{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.5"
    },
    "toc": {
      "nav_menu": {},
      "number_sections": true,
      "sideBar": true,
      "skip_h1_title": false,
      "title_cell": "Table of Contents",
      "title_sidebar": "Contents",
      "toc_cell": false,
      "toc_position": {},
      "toc_section_display": true,
      "toc_window_display": false
    },
    "colab": {
      "name": "softmax-regression-mlxtend-1.ipynb",
      "provenance": [],
      "toc_visible": true
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "U9KvNHXcHXHI",
        "colab_type": "text"
      },
      "source": [
        "Deep Learning Models -- A collection of various deep learning architectures, models, and tips for TensorFlow and PyTorch in Jupyter Notebooks.\n",
        "- Author: Sebastian Raschka\n",
        "- GitHub Repository: https://github.com/rasbt/deeplearning-models"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ru8sScr1Hox7",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "!pip install -q IPython\n",
        "!pip install -q ipykernel\n",
        "!pip install -q torch\n",
        "!pip install -q watermark\n",
        "!pip install -q matplotlib\n",
        "!pip install -q tensorwatch\n",
        "!pip install -q sklearn\n",
        "!pip install -q pandas\n",
        "!pip install -q pydot\n",
        "!pip install -q hiddenlayer\n",
        "!pip install -q graphviz"
      ],
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "K6-OArxHHXHL",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 121
        },
        "outputId": "a4529006-8a2c-4710-fc6a-3dd3ff8295b9"
      },
      "source": [
        "%load_ext watermark\n",
        "%watermark -a 'Sebastian Raschka' -v -p torch"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Sebastian Raschka \n",
            "\n",
            "CPython 3.6.9\n",
            "IPython 5.5.0\n",
            "\n",
            "torch 1.5.1+cu101\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eDvAElZlHXHS",
        "colab_type": "text"
      },
      "source": [
        "- Runs on CPU or GPU (if available)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JmhpE9P2HXHU",
        "colab_type": "text"
      },
      "source": [
        "# Softmax Regression with MLxtend's plot_decision_regions on Iris"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6v0OIrr7HXHV",
        "colab_type": "text"
      },
      "source": [
        "Implementation of softmax regression (multinomial logistic regression) and demonstration for how to use PyTorch models with MLxtend's [`plot_decision_regions`](http://rasbt.github.io/mlxtend/user_guide/plotting/plot_decision_regions/) function"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RR_BZIGlHXHW",
        "colab_type": "text"
      },
      "source": [
        "## Imports"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Pnej-WFLHXHY",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from torchvision import datasets\n",
        "from torchvision import transforms\n",
        "from torch.utils.data import DataLoader\n",
        "import torch.nn.functional as F\n",
        "import torch\n",
        "import numpy as np"
      ],
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6vBQGT8hHXHc",
        "colab_type": "text"
      },
      "source": [
        "## Settings and Dataset"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CqVx_9obHXHd",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "##########################\n",
        "### SETTINGS\n",
        "##########################\n",
        "\n",
        "# Device\n",
        "device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n",
        "\n",
        "# Hyperparameters\n",
        "random_seed = 0\n",
        "learning_rate = 0.05\n",
        "num_epochs = 10\n",
        "batch_size = 8\n",
        "\n",
        "# Architecture\n",
        "num_features = 2\n",
        "num_classes = 3"
      ],
      "execution_count": 4,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "g3CvByT-JWH4",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 227
        },
        "outputId": "923e7c64-4634-43ed-ba2e-c5c68a13d44e"
      },
      "source": [
        "!mkdir -p data\n",
        "!wget -O \"data/iris.data\" https://raw.githubusercontent.com/DeepSE/deeplearning-models/master/pytorch_ipynb/data/iris.data"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "--2020-06-27 09:45:35--  https://raw.githubusercontent.com/DeepSE/deeplearning-models/master/pytorch_ipynb/data/iris.data\n",
            "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.0.133, 151.101.64.133, 151.101.128.133, ...\n",
            "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.0.133|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 4551 (4.4K) [text/plain]\n",
            "Saving to: ‘data/iris.data’\n",
            "\n",
            "\rdata/iris.data        0%[                    ]       0  --.-KB/s               \rdata/iris.data      100%[===================>]   4.44K  --.-KB/s    in 0s      \n",
            "\n",
            "2020-06-27 09:45:35 (50.1 MB/s) - ‘data/iris.data’ saved [4551/4551]\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qR5x6_GyHXHg",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 69
        },
        "outputId": "ecdb75fe-801f-4d48-8a3d-b2f4c9c3312d"
      },
      "source": [
        "##########################\n",
        "### DATASET\n",
        "##########################\n",
        "\n",
        "data = np.genfromtxt('./data/iris.data', delimiter=',', dtype=str)\n",
        "X, y = data[:, [2, 3]], data[:, 4]\n",
        "X = X.astype(float)\n",
        "\n",
        "d = {'Iris-setosa': 0, 'Iris-versicolor': 1, 'Iris-virginica': 2}\n",
        "y = np.array([d[x] for x in y])\n",
        "y = y.astype(np.int)\n",
        "\n",
        "print('Class label counts:', np.bincount(y))\n",
        "print('X.shape:', X.shape)\n",
        "print('y.shape:', y.shape)\n",
        "\n",
        "# Shuffling & train/test split\n",
        "shuffle_idx = np.arange(y.shape[0])\n",
        "shuffle_rng = np.random.RandomState(123)\n",
        "shuffle_rng.shuffle(shuffle_idx)\n",
        "X, y = X[shuffle_idx], y[shuffle_idx]\n",
        "\n",
        "X_train, X_test = X[shuffle_idx[:70]], X[shuffle_idx[70:]]\n",
        "y_train, y_test = y[shuffle_idx[:70]], y[shuffle_idx[70:]]\n",
        "\n",
        "# Normalize (mean zero, unit variance)\n",
        "mu, sigma = X_train.mean(axis=0), X_train.std(axis=0)\n",
        "X_train = (X_train - mu) / sigma\n",
        "X_test = (X_test - mu) / sigma"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Class label counts: [50 50 50]\n",
            "X.shape: (150, 2)\n",
            "y.shape: (150,)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3i1C6ZQdHXHl",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "##########################\n",
        "### Data Loaders\n",
        "##########################\n",
        "\n",
        "from torch.utils.data import Dataset\n",
        "from torch.utils.data import DataLoader\n",
        "\n",
        "\n",
        "\n",
        "class MyDataset(Dataset):\n",
        "\n",
        "    def __init__(self, X, y):\n",
        "    \n",
        "        self.X = torch.tensor(X, dtype=torch.float32)\n",
        "        self.y = torch.tensor(y, dtype=torch.int64)\n",
        "\n",
        "    def __getitem__(self, index):\n",
        "        training_example, training_label = self.X[index], self.y[index]\n",
        "        return training_example, training_label\n",
        "\n",
        "    def __len__(self):\n",
        "        return self.y.shape[0]\n",
        "    \n",
        "    \n",
        "train_dataset = MyDataset(X[:100], y[:100])\n",
        "test_dataset = MyDataset(X[100:], y[100:])\n",
        "\n",
        "\n",
        "train_loader = DataLoader(dataset=train_dataset,\n",
        "                          batch_size=batch_size,\n",
        "                          shuffle=True, # want to shuffle the dataset\n",
        "                          num_workers=4) # number processes/CPUs to use\n",
        "\n",
        "test_loader = DataLoader(dataset=test_dataset,\n",
        "                         batch_size=batch_size,\n",
        "                         shuffle=False,\n",
        "                         num_workers=4)"
      ],
      "execution_count": 7,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Bsq7v8FFHXHp",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "##########################\n",
        "### MODEL\n",
        "##########################\n",
        "\n",
        "class SoftmaxRegression(torch.nn.Module):\n",
        "\n",
        "    def __init__(self, num_features, num_classes):\n",
        "        super(SoftmaxRegression, self).__init__()\n",
        "        self.linear = torch.nn.Linear(num_features, num_classes)\n",
        "        \n",
        "        self.linear.weight.detach().zero_()\n",
        "        self.linear.bias.detach().zero_()\n",
        "        \n",
        "    def forward(self, x):\n",
        "        logits = self.linear(x)\n",
        "        probas = F.softmax(logits, dim=1)\n",
        "        return logits, probas\n",
        "\n",
        "model = SoftmaxRegression(num_features=num_features,\n",
        "                          num_classes=num_classes)\n",
        "\n",
        "model.to(device)\n",
        "\n",
        "##########################\n",
        "### COST AND OPTIMIZER\n",
        "##########################\n",
        "\n",
        "optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)  "
      ],
      "execution_count": 8,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "km_1S5tXJ69b",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 183
        },
        "outputId": "21725e08-2ef1-4679-adda-c8786428438f"
      },
      "source": [
        "import hiddenlayer as hl\n",
        "features, _ = next(iter(train_loader))\n",
        "print(features.shape)\n",
        "hl.build_graph(model, features)"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "torch.Size([8, 2])\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<hiddenlayer.graph.Graph at 0x7f68cf951dd8>"
            ],
            "image/svg+xml": "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n<!-- Generated by graphviz version 2.40.1 (20161225.0304)\n -->\n<!-- Title: %3 Pages: 1 -->\n<svg width=\"304pt\" height=\"108pt\"\n viewBox=\"0.00 0.00 304.00 108.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(72 72)\">\n<title>%3</title>\n<polygon fill=\"#ffffff\" stroke=\"transparent\" points=\"-72,36 -72,-72 232,-72 232,36 -72,36\"/>\n<!-- /outputs/3 -->\n<g id=\"node1\" class=\"node\">\n<title>/outputs/3</title>\n<polygon fill=\"#e8e8e8\" stroke=\"#000000\" points=\"54,-36 0,-36 0,0 54,0 54,-36\"/>\n<text text-anchor=\"start\" x=\"14\" y=\"-15\" font-family=\"Times\" font-size=\"10.00\" fill=\"#000000\">Linear</text>\n</g>\n<!-- /outputs/4 -->\n<g id=\"node2\" class=\"node\">\n<title>/outputs/4</title>\n<polygon fill=\"#e8e8e8\" stroke=\"#000000\" points=\"160,-36 106,-36 106,0 160,0 160,-36\"/>\n<text text-anchor=\"start\" x=\"116\" y=\"-15\" font-family=\"Times\" font-size=\"10.00\" fill=\"#000000\">Softmax</text>\n</g>\n<!-- /outputs/3&#45;&gt;/outputs/4 -->\n<g id=\"edge1\" class=\"edge\">\n<title>/outputs/3&#45;&gt;/outputs/4</title>\n<path fill=\"none\" stroke=\"#000000\" d=\"M54.0225,-18C66.6333,-18 81.8509,-18 95.5484,-18\"/>\n<polygon fill=\"#000000\" stroke=\"#000000\" points=\"95.6583,-21.5001 105.6582,-18 95.6582,-14.5001 95.6583,-21.5001\"/>\n<text text-anchor=\"middle\" x=\"80\" y=\"-21\" font-family=\"Times\" font-size=\"10.00\" fill=\"#000000\">8x3</text>\n</g>\n</g>\n</svg>\n"
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AtGLpa5aHXHu",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 363
        },
        "outputId": "61201d95-4259-4ad5-b0e7-7a119ef61db5"
      },
      "source": [
        "# Manual seed for deterministic data loader\n",
        "torch.manual_seed(random_seed)\n",
        "\n",
        "\n",
        "def compute_accuracy(model, data_loader):\n",
        "    correct_pred, num_examples = 0, 0\n",
        "    \n",
        "    for features, targets in data_loader:\n",
        "        features = features.to(device)\n",
        "        targets = targets.to(device)\n",
        "        logits, probas = model(features)\n",
        "        _, predicted_labels = torch.max(probas, 1)\n",
        "        num_examples += targets.size(0)\n",
        "        correct_pred += (predicted_labels == targets).sum()\n",
        "        \n",
        "    return correct_pred.float() / num_examples * 100\n",
        "    \n",
        "\n",
        "for epoch in range(num_epochs):\n",
        "    for batch_idx, (features, targets) in enumerate(train_loader):\n",
        "        \n",
        "        features = features.to(device)\n",
        "        targets = targets.to(device)\n",
        "            \n",
        "        ### FORWARD AND BACK PROP\n",
        "        logits, probas = model(features)\n",
        "        \n",
        "        # note that the PyTorch implementation of\n",
        "        # CrossEntropyLoss works with logits, not\n",
        "        # probabilities\n",
        "        cost = F.cross_entropy(logits, targets)\n",
        "        optimizer.zero_grad()\n",
        "        cost.backward()\n",
        "        \n",
        "        ### UPDATE MODEL PARAMETERS\n",
        "        optimizer.step()\n",
        "        \n",
        "        ### LOGGING\n",
        "        if not batch_idx % 50:\n",
        "            print ('Epoch: %03d/%03d | Batch %03d/%03d | Cost: %.4f' \n",
        "                   %(epoch+1, num_epochs, batch_idx, \n",
        "                     len(train_dataset)//batch_size, cost))\n",
        "            \n",
        "    with torch.set_grad_enabled(False):\n",
        "        print('Epoch: %03d/%03d training accuracy: %.2f%%' % (\n",
        "              epoch+1, num_epochs, \n",
        "              compute_accuracy(model, train_loader)))"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch: 001/010 | Batch 000/012 | Cost: 1.0986\n",
            "Epoch: 001/010 training accuracy: 44.00%\n",
            "Epoch: 002/010 | Batch 000/012 | Cost: 0.9489\n",
            "Epoch: 002/010 training accuracy: 66.00%\n",
            "Epoch: 003/010 | Batch 000/012 | Cost: 0.8236\n",
            "Epoch: 003/010 training accuracy: 72.00%\n",
            "Epoch: 004/010 | Batch 000/012 | Cost: 0.8275\n",
            "Epoch: 004/010 training accuracy: 95.00%\n",
            "Epoch: 005/010 | Batch 000/012 | Cost: 0.6650\n",
            "Epoch: 005/010 training accuracy: 72.00%\n",
            "Epoch: 006/010 | Batch 000/012 | Cost: 0.6465\n",
            "Epoch: 006/010 training accuracy: 78.00%\n",
            "Epoch: 007/010 | Batch 000/012 | Cost: 0.3825\n",
            "Epoch: 007/010 training accuracy: 95.00%\n",
            "Epoch: 008/010 | Batch 000/012 | Cost: 0.4917\n",
            "Epoch: 008/010 training accuracy: 77.00%\n",
            "Epoch: 009/010 | Batch 000/012 | Cost: 0.6003\n",
            "Epoch: 009/010 training accuracy: 81.00%\n",
            "Epoch: 010/010 | Batch 000/012 | Cost: 0.5938\n",
            "Epoch: 010/010 training accuracy: 93.00%\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "W_73u4CKHXH0",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "c6438dce-4165-4d11-8d83-7c7c97153219"
      },
      "source": [
        "print('Test accuracy: %.2f%%' % (compute_accuracy(model, test_loader)))"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Test accuracy: 84.00%\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GxXiXfcmHXH5",
        "colab_type": "text"
      },
      "source": [
        "## Creating a ModelWrapper class for plot_decision_regions"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "f8zywRymHXH6",
        "colab_type": "text"
      },
      "source": [
        "All we need is a simple class that implements a `predict` method. That's it."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qmAatckJHXH6",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "class ModelWrapper():\n",
        "    def __init__(self, model, device):\n",
        "        self.model = model\n",
        "        self.device = device\n",
        "        \n",
        "    def predict(self, X):\n",
        "        features = torch.tensor(X, dtype=torch.float32, device=self.device)\n",
        "        logits, probas = self.model(features)\n",
        "        _, predicted_labels = torch.max(probas, 1)\n",
        "        \n",
        "        return predicted_labels.numpy()"
      ],
      "execution_count": 12,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ucGUxVBAHXH_",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "mymodel = ModelWrapper(model, device=torch.device('cpu'))"
      ],
      "execution_count": 13,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "su3RY2PDHXIG",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 324
        },
        "outputId": "cbb9b8b0-48b7-40c5-a0c2-bae8982b1516"
      },
      "source": [
        "%matplotlib inline\n",
        "import matplotlib.pyplot as plt\n",
        "from mlxtend.plotting import plot_decision_regions\n",
        "\n",
        "plot_decision_regions(X, y, mymodel)\n",
        "plt.show()"
      ],
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/mlxtend/plotting/decision_regions.py:244: MatplotlibDeprecationWarning: Passing unsupported keyword arguments to axis() will raise a TypeError in 3.3.\n",
            "  ax.axis(xmin=xx.min(), xmax=xx.max(), y_min=yy.min(), y_max=yy.max())\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD8CAYAAAB0IB+mAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd3xUZfb48c+ZzGTSC0kIaZBA6EgRKRYUC4iKwCoiRVzXVbDr6n6t+9N1q1t0F8uqrGJbFFFsi6CggnVVighK7wHSe2/z/P6YJCYhPZPMJDnv14vX5t6597knLJ7cPOcpYoxBKaVU92dxdwBKKaU6hyZ8pZTqITThK6VUD6EJXymleghN+Eop1UNowldKqR6i3QlfRHxE5FsR+V5EfhSRhxu45hoRSReRbVV/rmvvc5VSSrWO1QVtlALnGWMKRMQGfCEia40xX9e77nVjzC0ueJ5SSqk2aHfCN86ZWwVVh7aqPzqbSymlPIwr3vARES9gC5AIPGWM+aaByy4XkbOBvcCvjDFJDbSzCFgE8Md//GHsubMnuSI8pZTqMU6PPVsa+0xcubSCiIQAbwO3GmN+qHU+DCgwxpSKyGLgSmPMeU219V3KJrMnc6fLYlNKqZ5g7vCfN5rwXTpKxxiTA2wAptU7n2mMKa06fA4Y68rnKqWUap4rRulEVL3ZIyK+wBRgd71romodzgB2tfe5SimlWscVffhRwEtV/fgWYKUxZrWI/A7YbIx5D7hNRGYAFUAWcI0LnquUUqoVXNqH70rah6+Uchcxgj9B2C12hEa7xN3GYCh1lFJIHkbq5vCm+vBdMkpHKaW6E3+CCPILAovBA/M9GLA77FAEBeS2+DZdWkEppeqxW+yem+zBGZfFOONsBU34SilVjyCem+yrCa3ubtKEr5RSPYQmfKWU8kDfbPyWq867hvnnXM3yf73mkjY14SullIeprKzknw8+wV9f/BMvrX+ej9/bwOF9R9rdro7SUUqpdrhh9l3k5BSddD4kxI9n3ny0TW3u2raHmH7RRPeNBuC8SyfzxboviR/Yr12xasJXSql2yMkpYtAN/zzp/N5n7mhzmxmpGfSO7l1zHBEVwa5tu5u4o2W0S0cppXoITfhKKeVhwiPDSTuRVnOcnpxOeGRYu9vVhK+UUh5myKjBHDt8nOSkZMrLyvnkvxs5c8oZ7W5X+/CVUsrDWK1e3PG7W/n11ffiqHRw8ZxpJAyKb3+77Q9NKaV6rpAQvwYLtCEhfu1qd+K5E5h47oR2tVGfJnyllGqHtg69dAftw1dKqR5CE75SSvUQrtji0EdEvhWR70XkRxF5uIFr7CLyuojsF5FvRCS+vc9VSinVOq54wy8FzjPGjAJGA9NEZGK9a34JZBtjEoF/AH9xwXOVUkq1QrsTvnEqqDq0Vf2pv2/iTOClqq/fBM4XEU9fbVoppboVl/Thi4iXiGwD0oD1xphv6l0SAyQBGGMqgFzgpGljIrJIRDaLyOZVr7ztitCUUqpLeuT//sbMsbO5Zup1LmvTJQnfGFNpjBkNxALjRWREG9tZaow5zRhz2uULf+aK0JRSqku6aPaF/O2lP7u0TZeO0jHG5AAbgGn1PjoOxAGIiBUIBjJd+WyllHKnnKxcHrr+fnKzW76peFNGTRhJYHCgS9qq5opROhEiElL1tS8wBai/jud7wM+rvp4NfGKMqd/Pr5RSXdb6N9ZQkbSXdSvXuDuURrniDT8K2CAi24FNOPvwV4vI70RkRtU1zwNhIrIfuBO41wXPVUopj5CTlcumNetZcnkUm9asd9lbvqu1e2kFY8x2YEwD5x+s9XUJcEV7n6WUUp5o/RtruDRRGBjpw6WJRaxbuYYrFs9zd1gn0Zm2SinVDtVv9wvGBgGwYGyQx77la8JXSql2qH67DwtwdpiEBVi5NFHa3Zf/8K1/5KbLbuPowSRmT5zL+6+vbXesulqmUkq1w/dfbWXDiRJe236izvleGVvb1a3z0BMPtDe0k2jCV0qpdvjDS39zdwgtpl06SinVQ2jCV0qpHkITvlJK9RCa8JVSqofQhK+UUj2EjtJRSikPlHYijT/e+ReyM7IRES6ddwmzr72sXW1qwldKKQ/kZfXi5t/cwKARAykqKOL6S2/ktEljiR/Yr81tasJXSql2+nrjt6x6dRXJSSlExfXh8vmXM3Hy+Ha1GdY7jLDezn2i/AL86DegL+kpGZrwlVLKXb7e+C3/fnYp8TOj6ZswgpxD+fz72aUA7U761ZKTUti3cz/DRg9pVztatFVKqXZY9eoq4mdG0ysxGIuXhV6JwcTPjGbVq6tc0n5RYTEP3vgwtz54E/6B/u1qSxO+Ukq1Q3JSCiEJdXemCkkIJDkppd1tV5RX8OANv+WCWedz9rRJ7W5PE75SSrVDVFwfcg7l1zmXcyifqLg+7WrXGMNf7vk7/RL7ceV1s9vVVrV29+GLSBzwMhAJGGCpMWZJvWsmA+8Ch6pOvWWM+V17n62U8kzbvtjOByvXkX4ig4jocKbNmcros0a26p5Bwwey98d9rWrDHS6ff7mzz36m880+51A+h989wfWLF7Wr3R2bf2DdWx/Rf0gCv7xoMQDX330tE8+d0OY2XVG0rQDuMsZsFZFAYIuIrDfG7Kx33efGmOkueJ5SyoNt+2I7r7+wkvhZUcTHDyX3cAGvv7ASoNGEXf+eg+uO8eH7HzJsQSLjhrWsDXepLsyuenUVe5KOEBXXh+sXL2p3wXbkuFP49PBHrgixhiu2OEwGkqu+zheRXUAMUD/hK6V6gA9WriN+VhShA5w7QIUOCIJZzvONJev692TszCZhbhQS7MDiZWlRG+40cfJ4l43I6Ugu7cMXkXic+9t+08DHp4vI9yKyVkSGN3L/IhHZLCKbV73ytitDU0p1kvQTGQTHB9Q5FxwfQPqJjBbfU5ReQnBiABXl5S1uQzXPZQlfRAKAVcAdxpi8eh9vBfoZY0YBTwDvNNSGMWapMeY0Y8xply/8matCU0p1oojocHIPF9Q5l3u4gIjo8Bbf4xfhQ+7+Aqw2W4vbcCWDcVYkPZmpirMVXDLxSkRsOJP9cmPMWyfFVesHgDFmjYj8S0TCjTH641qpTtaWgmpz3nr2Xda/8zElhSV4eVnxS/Zh+NUDCI4PIPdwAYffSebKX8xp9P5pc6Y6++hnOd/kw4eFcmjFcYYtSMRR6WhRG65U6ijF7rCDxYB0yiNbxwAOodRR2qr4XDFKR4DngV3GmMcauaYPkGqMMSIyHudvFpntfbZSqnXaUlBtzlvPvsuH739IwsIoghMDyN1fwL5lx9j7UhLgfHu/8hdzmmy/+rMPVq5j74kkIqLDufCSC9n75T42vbGrRW24UiF5UAR2ix3xwIxvMJQ6Sp1xtoIr3vDPBBYCO0RkW9W5+4G+AMaYZ4DZwI0iUgEUA3ONMZ7+C5NS3U5bCqrNWf/OxyQsjCJ0iHPyUeiQQAZeG8uRV9J5eu3jLW5n9FkjPaYga8RQQC4Fnpyl2vBzyBWjdL5o7tHGmCeBJ9v7LKVU+6SfyCA+fmidc8HxAew9kdTmNksKnQXWOm0mBlBS2PY2VcfQmbZK9SBtKag2x8ffWWCt0+b+Anz8fdrcpuoY4qk9K9+lbDJ7MnUov1KutO2L7bzy9H8ImxyIPcKL0vRKUtZlE+AdSGlpaYMzXJub8VrThz/3pz78g68mE2ANRKzS4D0tKRx3RHG5LTwljpaaO/znjfa46PLISvUwlSUOUjZkUlpQjsVLKMsvZ8D1sUQOCztphmtLZrxetngmAOtf+ZiSwiRs3jZsvjYG/7JvzSid2ve0pHDcEcXltvCUOFxFu3SU6kE+WLmOodckcNZvxnL+IxPx7eXDwGtja2a01p/h2tCM1/hZUXywcl2ddi9bPJOn1z7OC58tpf+wBEYuGkTogKAG76ldOG6szZZc01l/X54Qh6towleqB2luRmtzx9D6WbP172nJTNy2zNbtCJ4Sh6towleqB2luRmtzx9D6WbP172lJ4bgjistt4SlxuIomfKV6kGlzpnL4nWSyD+ThqHRUzWhNxuRaWnScfSCPw+8kM23O1DrtbvtiO4/c9nfumn0veZn57F1xpOYZ2Qfy2LviCHmZ+Y1+Xr/N+nE29tz2qh33I7f9nW1fbG/y76uj4ugsOkpHqR6muXXnWztKp3Zhs7pIu+vFQwQEBFBcWIKvvw8FBQUMvSahwc/dNUqnobirl29o7YgiT9LUKB1N+Eqpdnnktr8TOsW3ZvYuQPaBPLLXF3Pv479u9nN38dS42quphK9dOkqpdnFFkdYdPDWujqQJXynVLq4o0rqDp8bVkbRLRynVpIb6sIGac3a7ndy8HLwChdKCcuwBNsqzHIRF9mpXH35Hfy8NxdVQH35XozNtlVJt0tBM01ee/g+VJQ6GXpNQs/9sxoZy+pzTG79Ib4pSy0hanYbfSBsjpiaQe7iAvSuKOLIijdLSJOx2O14+FuJmRzQ4E7czv5facXX2EszuoF06SqlGNTTTNGxyIA57Rc25jJ3Z9J8fRXC/AOIGxRHcL4D+86LI2Jldc8+guf0ICgvk0TcfISgskEFz+3X67NWGvpfacd37+K+7dbIHTfhKqSY0VNi0R3hRWvDTzNv6s3EryssJTgygKL2k5hpPKOL2xCJtfa7Y8SoOeBmIxLnx1lJjzJJ61wiwBLgYKAKuMcZsbe+zlVJ1NTfGvn7/e3Pj7KsLm7WHLpamV2IP+GnmbfVsXL9evgBYbTbnccRPyyM3VMSt3aariqVNjZlv6LmpOzMpKizirtn3dokx9u3lijf8CuAuY8wwYCJws4gMq3fNRcDAqj+LgKdd8FylVC3VfdShU3wZ99BQHP1K+fD9D/E908K4h4YSOsWXV57+Dy/+4+Umr3n9hZU1M04bmmmauTEfS6m10dm6JtfCoRXJhA8LbXB2akfNXq3//Tf3vSTvSGfn8v3EXRDZ4PXdkSt2vEoGkqu+zheRXUAMUHuIzUzg5aptDb8WkRARiaq6VynlAvW3L2xopcuwyYGkbMhs8praWx42tNfswhuvOulc/f1nL7zkQvb+uI9ND5+8H21DbbqiWNrc9o31n1tUWES/C2JIvKhvg9d3Ry4dpSMi8cAY4Jt6H8UAtfc7O1Z1rk7CF5FFOH8D4IG/3seIiwe7MjylurX62xc6+9ZjKE79qS+94f73utfU3/Kwsb1m25MUO2L/2pZs31j7uXfNvpf+U2ObvL67cVnRVkQCgFXAHcaY1m2lXsUYs9QYc5ox5rTLF/7MVaEp1SM0txImNN7/3prVMD1VaydS9cSJVy55wxcRG85kv9wY81YDlxwH4modx1adU0q5yLQ5U53j2Wc531TDh4Vy8NXjxF3am/KSsprtDIuzytjw4DeUF1aAgfzlhYy4eiCOSkedSVHuLmS2dtGy+t9/7YlUrri+O3DFKB0Bngd2GWMea+Sy94BbRGQFMAHI1f57pVyrfh+13W7HJjbSP82pmQFbVlBBJZXEXRCOT6Q3JallHH4jlaNvpHGUNLdNiqqvLVsLtrY20FG1BE/W7qUVROQs4HNgB+CoOn0/0BfAGPNM1Q+FJ4FpOIdl/sIYs7mpdnVpBaXap6HVIHdt2EvKZ9mMviex5lz27nyOvJLO02sf95gVJD0ljq6oQ5dWMMZ8ATT6gKprDHBze5+llGq5hoqYPpHeVBRV1jkXnBhASWFSo/e4o5DpKXF0NzrTVqluqqGiZElqGVY/rzrncvcX4OPv0+g9urVg96GLpynVhb317Luse/sjivOL8Q30JTY2huPHT1BSWILN24btkJWRiwbV9MdnbiykOLWUbX/ZT0VRJVY/L8oyy7nkiouBlhcy33r2Xda/8zElhSX4+PswYsxw8gvym9wVq7kCbP2VLFNeTGlwJcvWtKnq0oSvVBf11rPv8uH7HxJ/VR8C+tnJ2VXE4XcO0/uMUOIuTCB3fwEHX01mz/NHEasQER3O5AvOYeNHnxJ2tn9N0TZzYyH9hycALStkVj83YWEUwYkBZO3K47uV3xF3Zh/GXX9ygbUlBdjWrmTZlqKu0oSvVJe1/p2PSVgYRUCCD95eYO9tI2FBH5LXZ2GxCqFDAuk/H2dB9p3HAWcxdPjVA34qhg6D7L55dWaXNjcpqvq5oUMCAfCJ8Kb//ChOrM1k6KwBJ81YbW4GLDQ8S3bQ3H5VRdqHT4qhJW2qk2kfvlJdVElhCQHxvngJWAQwhqD+fpRmlNVc4yzI/jSL1hUrRpYUOlfHrOaocBA0wI+SrFrPbeXqmK2NS1e+bBtN+Ep1UXY/O3n7C7BW/VcsIuQdKMIe7l1zTe2CLLimGOrj75ydW81itZB3oAifXrWe28otDnWWbOfQhK+Uh9j2xXYeue3v3DX7Xh657e/NrtoY3y+Ww6+nkLOvkMoKQ2l6GYdeTSEwwRdHhSF7dz6HViQzadokltz6GPk5BS5ZqXLKrPM5tCKZ7N35OCoMJellHHw1mYhhvdq8OmZr4+qoFTe7O+3DV8oDtKUIaRew51vY+cxxKisNXl6CpRRSP8km5eNsfPx9uHDWhfjavMg8cZgv3trIRddOB9o3u/SyxTMBWP/Kx5QUJuHj78OYMWPIT8lv8+qYOku2c+gm5kp5gI6aWZqXnc/Sm/7EU9P9uXl1IYuffoDAkIDmb1RdVlMzbbVLRykP0FFFyC/f/pQZiRYGRtqZkWjhi7c2tqs91bVpwlfKA3REETIvO58f13/JvLHOHyTzxgbw4/ovyc8paOZO1V1pwlfKA1QXITP3ZpN6KIXMvTmNFiHzsvNrirBNqX67DwtwlurCAqw1b/ktbcNVWluQVh1Di7ZKeYDqYuPyJa+Sk5JJSJ8wFtw+v8Ei5Jdvf4qlXhG2Ibu//ZFvU0p5bUdqnfNB6T8CtKgNV9BZsZ5DE75SHqL/8AQircKbN/Tj5tWFDBjR/6RrqrtpnrqsNzev/pKzLpvcaBH2V8/c0+D5mkJuC9pwBZ0V6zm0S0cpD9GSAqsrirCdXcjVWbGew2MT/quP/Ye//uJv7Ny0m91b9pCalObukJTqMC0psLqiCOuOQq7OivUcrtrTdhkwHUgzxoxo4PPJwLvAoapTbxljftdUm5ctfpCKkEQ2v74aRDh27AgDRvXD29u52fKAUwcycvIoV4SvVIfIy87nhQf/zbW/X9Rsl0n1W/eWQwVc/8pxlv08lhmJFj5e/iFHdh/h2t8vqrkmxEdIPZpKeHREzRt67X74E4eS+dsv/8TdLzxAalJanSWEw0OCGRpYxuJVyRzNKqdvLxtDA20d2pffE/eO9VQumXglImcDBcDLTST8XxtjWvwv6n8HMs0Px3NrjvNzstj12Ts1W2tt3riGoGAbFovgF+rPvAcW4GX1argxpdxg7bLV7H9/HYmXTG02mf7jhr+Ql5JOclIG/UMtHMx2EBUXTrnDQqSthMRLprL72x/JS0mnuKCYiqJirH6++Ab4EtQnok5//ZKb/07Z3l0UhcdS4ldJ/KyomkT73bM7KS+uIOHyPvhH2SlMLuXY++lEBoTx8Ou/77C/C127vvN06BaHAMaYz0Qk3hVtNSYwpBfjZ1xbczx62gIyko8BsOPjN/j7NY9i8bJQVlHOjJtn4Bfo/I8hOiGqI8NSqkGtKa6Cs8D65dpveO8Pz7Bspi8zVxRx9tUz2PLGWh6b7mxj8dMPYIypmjkb1eDM2ROHkkn9YQ9vzglg8orjjPrVKXWKpX1n9iFlQyaDJlftaTsYIuJ6k72+uEP/Pppbcll1js4cpXO6iHwPnMD5tv9j/QtEZBGwCODuPzxK4qRZjTZm87YT1W8AAFHX3kv1aOUTB37k8+WPYfXyIjU9jegB4YT36YXVx5tzF5yP1aYDk1THq1sYLW5Rl8mK3z/PopE2RvXx4qpTbDz9yEvccWGfOm0ATbb7xt+XM3+ElZFRVuxWcNjL6jzDHuFFaUF5nXO6V2zP0VnZbyvQzxhTICIXA+8AA+tfZIxZCiyFk7t0Wip6wHCufPB5AMrLStmy+mUqyotJ2X+Mx657jF7hwQCI1cK83yzAL9Cvzd+UUg2pfrv/zZXON+t5YwOY93rTb/lfrv0GPynn5vHOf483j7fxnx3lxAQ4atq48tXPcRj4zYKQBtutfru/4VrnM4ZFeJFxKJdeMWHYqmpfpemV2ANsdZ6tBdSeo1MSvjEmr9bXa0TkXyISbozp0HFZNm87Ey+7vuY47dhhSkucv7om79nKPxYtwWaz0jcujCgP+wcvoQFMmncuIo12x6lO1tIibFMzXOu/5VcXWMuKi7lhlI2oQGcdKirA+Zb/8HspnBFRTnh0BFOiy/ghtZLKykDmPXWIx6/qx4xECx8t/5Cju49QUVrG/BFW+gQ6B9/931hvFq/P4KjdTsLEvlV72uZjKbWSfSDPpQVU7aPvGly2WmZVH/7qRoq2fYBUY4wRkfHAmzjf+Bt9eFvf8FvLGMOO9a+Tm3IY8k5w0wWDmDA83u2J9vMfj/DOpn10l3x/NK+YYRePQ+i639DX73/Jsa++ZcD5ZzH31/Mbva66AFtf/eIq/FRgPZBViY/15L+bkgpDgN2C1c+XitJyyioNFovQx15BSqmV0PAgyh1CpK2UvcnF2HDUub+0wlBq9yI4MqwmEQMuTc61Z9LW/yGiSb/zdXjRVkReAyYD4SJyDHgIsAEYY54BZgM3ikgFUAzMbSrZdyYRYeTUuQAU5uXwytbPeXrdWuJ7BzF9fH/GDYp2S1yThvdj0vB+bnl2R8jKK2TT7mPuDqPN8gqLefvrbdxxehB/+uB/eFVaCO4V1OC1w8ee0mg76/69pubr3Ox89mzayYJTbBzKNZw1+0KCQgJrPi8tLuWzdz9h5hBv3t1dxjlzLqKkuIwNb37IpME2lu8oJ3HsKLZt/JYxQ7w5mmM4e9Z52H3sjT4/bZfz/4NTTxkGVWEWpuTgcDiwWNo2LUdn0nYdHrsefme94TeksqICR2Ul3723FJ+SdLwdJfzxqjPwsduav1l1S48tXwfHt3Dn2cE89lkuFX3GcPvc89vV5hX3P8MI61EemuzPwxsL+aGiL2/86Yaaz5es+AiOf8ftk4JY8nkexIzhi+3769zz/rEAFgy31bnm9rkXtCqObYdS+dfGHfj6+bbp+1jxxofETo6o81uxMYZjG9OZe8WFbWqzPQrLyjn72gt77Lr/Hf6G3914Wa14Wa2Mv+JWALLSTrDolRcpKynivP52Lh0/gKiq4q/q/jJyClj96SZWznG+fV99qj9zVm7hlzMnERbs36Y29xxJZefeQzx7bQB2q3DrRF8mLTvE0dRsBsb1JiOngA++2MrKOYHYrRauHRfAjFc2kZadU3PP/JF2Xt2exfzR8TXXzFm5lV/OPLtVcU0YHMuEwbFt+j4ADu3cic8YPyJqbd6SfiCPmHQfnlk4uc3ttlVeYTF/fGU9FZ75LtsqxaXlOGLC6Tvq5HWVGjW88Y804bdAr97RTLrmfgAO79zCnSvfY3hoOcF+dm68ZAxWnfDV5WTkFLD4kf+w9L6FzSbHl9//iumJFsKrirDhAVamJ1r47XP/ZfkH3/DBktsZPyyePUdSmXb7EtY9cQcD43qfdAzUnBsY25v5I6xEBlgoKa+gT4AX80dYuf2x1/C12xk9MJbpic4ulstfSGLpnGiCLQWcOdiLqADnv7d1+8pZMNJGSWEehEbUxPXS6i+5c0Hn7e16x8zJ3L3ivzATwuIDyDxcwMF3U/nr3Es7LYbagvx9+cvV7fvty5Ns3nOMpCPJLmlLu3TawBhDcWEBOWnHOfDB8wT7e3PRKZFcPH6Au0NTLfTY8nWsXv8p06ec02xynHHXk5xIO3lA2bH0PKL9HRi/Xnz3yoP87O6nOHpwP337J/L2X28+6RioOXc4x+BjE4zD4DAGiwhiESqNhWF9fMgq98ZqMWTlFxNiLSOnwpvcwhJsAl5ezh8EJeUObBZnHSqmd0hNXNG9w3nv0Vtc+LfVvLVf7+Sf727kUEoWCX16ccfMyVw0cVinxqCqnHGrdum4kojgFxCIX8AQom/6GwAffPQ6by77Hkd+GndcNJRAPzsDYiPcHKlqSHUXzdOXhXPj6k38fPqZTb7lN5Q8v95xiLn3/JNlM/2YtSKTNz7eyo7dB3hrjj+XrTzAh1/vqnO8LykNh8PUOlfE8j/fxn1LlvP0dD9uXF3EM7+5nhv+8O9Gj9/424Nt7kLqaBdNHKYJvgvw+u1vf+vuGBp0LLv4t2n5pe4Oo8Ui+48gbtQkwoZMZMOebD7anc0XX28hNMBOTFhg8w2oTvPMqo0MsqVw6TB/MvPL+D6lgtNHtu63s0t//QSXD6xgwSl20osc/PHt7VwzysrcU3zIKKzkT29v56rhUnO8/OvjfLRpJxf0ya8599iaXVwx1FoTx/MbDzAltqzR47bEqXqguAkPN/aRxy6P3FX5B4Uw4pxLOX32DcRe/hDPbinl9mVfcfcLn1FcWtZ8A6pDVb/dX32q80356lP9Wf3pJjJzC1vcxtc7DpGdmcnN47wBmDfChl0quGa083j2MBuOsmKuH+scHnnjeB+27zrAtp37uXG8DwA3jPMhOzOTM/s5R35dMtiHHbsPMH+0c6btvFF+7Nh9gOlDfNocp1L16Rt+B7LZ7cQOH0/vUefjFTOS599Yy9ubj5KRnk6wrxVvqxVvXdunU1W/3Z8/0JlY/bwtLX57zsgpYOFvl/Hymq+YPaiCs/paWfh2ETnFhmERXgyLsBARYOWZTcUMj7AwIcYLm5chxNeLpJxyyisMN1UtneDjZcgqdrByezGrtueTWVTJ8F6VjIuxEeBnp7CwkJLSUo7mCafH+9WJc2DfSBb+dhkXjBuKn493h/+dqS6miTd8Ldq6wYHvv6Ig/TgZO//H1OFhXHXeCE38naSxAmxLCp3Vhd5tSQV4e0F5pSEu2MKRHAd2q2AMWK0Wissq8fYSDCCARYRKYyivNPh6O0fYOBwOjIHSSkNCqBdJuQZvq+DlZaF3aCBp2flUVjpwYCEm/Kcuweje4Uw+dVCLC86qB2qiaKsJ340qyss4cWgPh9a9QESwD+cO7c2MiQs1UeAAABn4SURBVCetKac8QEZOAXPuXtJEQfWOOgXV+tfX/7yl1zQXR0vuUT1MEwlf+/DdyGrzpu+gUzjnlscYtvBPfJwTxfUvfMcvHv+Y/cfSScnMa74R1Smqx+IP7m1neqKFe558o87xS6u/bPL6+p+39BpX3KNUNU34HmTEBVcyfuFvGL3wtzz2nZV71qRx/4sbeWn99ySlZbs7vB6rfqG3foG1fkG1JYXhthSPXVFwVj2bJnwP5BcYxJiLruKsebcTOu3XHB0wl/tW7eGeFz+jsLhrF7JbIiOngMvvfcatiax2DNVv1eCc9bry+zzmj7BCuXOp7dozXKHxmbm138Zbck19bblHqdq0UujhQsIjAehz/e/Jy87gxpefwO4ljI+18bOJiQT62bF7d69F3V5+/yuyU5I6fYmAxmLYuHUvJ9JKefKrHEKsZXx2sBhvq/Di9nR6h5bU3BOdupc7F0ytuf7VHWl12qz+HGjRNfW15R6latOibRd1cOtnZB7dQ86RH7ns1D6EBPhw4WmJbl/Hv708oSjZUAzGGLfHpVSLNFG01XH4XVRoVD9ihoyl76nnctj04YcsL9787zrEOBgcG+bu8NrMFbNgOyKG7/cluT0upVpEZ9p2XzZvO7EDhjDirIuYeNMSPsqJ4oZlm7nuyY85np7Tpfr8PaEo2VAMb338De9u+FaLparLc0nCF5FlIpImIj808rmIyOMisl9EtovIqa54rjrZKRdcydirH2Lkwod58ON8Fr+6j//38mf896tdHE/PcXd4TWprUXLPkVQSZt3PvqS0Nl9TXaR9+q2NJ8UQYSvhnOiyk+L615sb6hSXPaHYrFRTXNKHLyJnAwXAy43saXsxcCtwMTABWGKMmdBUm9qH7zrpJ45SkJvN4c/eICGgnHtmjyPIv227G3Wkts6CbWgZ4tZeUz2LNqvCjlXq7gt7PCMfmxf0Dq27CF6FsdDLWloz47U1Sy4r1WE6enlkY8xnVZuYN2Ymzh8GBvhaREJEJMoY45pV/VWTIqL7EhHdl4Sho8jPyeKWV/6Bv7eFMVF2rp0yAotF2ryfqSu1ZQ33PUdST1qGuHqzkZZeU3e55CLe+NtdrZjx6lxiefqk0a1aclkpd+is/8pjgKRax8eqztUhIotEZLOIbH5nxUudFFrPEhjSi0nX/Z5Tr36Yg6Gnc93yfVz56Ees/XonX24/iKeO2mrMvU+9yfwRVkb2sTF/hJW7n3ij1de4YsZrczNvlfIE7n+tq8UYs9QYc5ox5rRZc3/u7nC6vQGnncuZC+7i3Jv/yie2c3g1tR/X/GMtdy/byLYDKe4Or1nVb+7VSw7fON45A7Z2P31z17hixmtzM2+V8hSdlfCPA3G1jmOrzikPYPO2M2jUeEafO4OzbnuKAVf9hSf+l8fipzeQlp3v7vAaVf3mXr3Ha1TVvrC13+Cbu8YVM17f31XQ5MxbpTxFZ820fQ+4RURW4Cza5mr/veeyWCxMnPsrSooK+b+3n6W8pJCB/oVMHRlD/+gwosKD3R0iAN/tSeLbsnKe/67u6CObd1KLr3HFjNfjGflYcDQ681YpT+GqUTqvAZOBcCAVeAiwARhjnhHn9M8ngWlAEfALY8zmptrUUTqeJfnIAXLSjpO8dT2JgeWEB/lwy6VjazbUVkp5CF0PX7mKMYaK8jLSk/ZzaP1LTBwQzA0XjXZ3WEqpaprwVUc5sHkDKTu/pigrlZvOjyfE35dRA2O6/Jo+SnVZmvBVR6soL2Pvls8oycumfN9nDIgKYdqp8YxJ7OPu0JTqWTThq85UVlpCRVkZ299/HnvBcf7f7FM9ptCrVLenCV+5S2lJMZtXPQXlJcT7FnLF6QPoHRpARL1lCpRSLqIJX3mC5EO7yTy6l9Sd33BalIVgfzvXTh2lI32UciVN+MqTOBwOCnKyyEk/wZGPXiIkwM700VFMHdvf3aEp1fVpwlee7vsPlpO1bwu3TR1A75AAEqLDdKSPUm2hCV91BRUV5ez8fDVlhfnIsS2M6R/OxMFRjBoQ5e7QlOo6NOGrrqYoP4/iwnz2blhJcHk6vl4OHpw7ER9799qwXSmX04SvurrstGR+XP0s/QPKuW7KMAL97B65iYtSbqcJX3UXJ/b/QMreLWQd2cu5CXaC/e1cMWkoVquXu0NTyjN09I5XqmP8+ZZ5FBScvDxxQEAg9z35mhsicr/oxBFEJ47AGEPascMczs1kzRP/ISLYl6kjo5l2mo70UaoxmvA9WEFBPv2ve+Kk8wefu9UN0XgWESEyLoHIuAQGjDgNgNUfLOftZVtxFOfw0BVjiIkI0ZE+StWiCV91G6OmLQCgMC+Hhz5aiXfGFs4dHsXQuDBO6a9r+iilCV91O/5BIUy8bBF52Rn8mJXBmi/X0uuTPfh5W7h/zgT8fLzdHaJSbuGShC8i04AlgBfwnDHmkXqfXwP8jZ+2NXzSGPOcK56tVGOCQsMJCg0ndsAQAHIy07jhxSfwsVo4o58v884Zgt1bh3mqnqPdCV9EvICngCnAMWCTiLxnjNlZ79LXjTG3tPd5PVlK0kEqKyvJzkjjgWum15zvyUXc1ggJ682k634PwM5vP2LBU+u5dHggYYG+TD0tEW+b/sKrujdX/AsfD+w3xhwEqNq3diZQP+GrVgoICKxToM3OSMMW0Auf8Fj6X/dozXkt4rbewPEXkDjufPbt/YEdhfm8uuQ1YsP8OHdEDBeNG+Du8JTqEK5I+DFAUq3jYzg3Kq/vchE5G9gL/MoYk9TANaqW+m/tD1wzvcFRO6ptRIR+g08BYPCpZwDw/gfL+e8Lm3CUFPCHeeMIC/bXkT6q2+is32H/C7xmjCkVkcXAS8B59S8SkUXAIoC7//AoiZNmdVJ4SjlVj/Qpys/jrveex6/gKLPHxxHXO5RBfXu7OTql2scVCf84EFfrOJafirMAGGMyax0+B/y1oYaMMUuBpaAzbZV7+QUGcdb8X5GTkcoHx4+QuuELwkp3EOjrzd2XjyfAz+7uEJVqNVck/E3AQBFJwJno5wLza18gIlHGmOSqwxnALhc8t8tpbubsjReOxnjVWiLAGEDAUUFYVBxZaSnk/PM6LDYfIi65veay3MyMFj9DtU5IeCQh4ZEMGjUegLzsDG584TGCfG2cFufLL6ac4uYIlWq5did8Y0yFiNwCfIhzWOYyY8yPIvI7YLMx5j3gNhGZAVQAWcA17X1uV9TczFnj5UXcLf+pOV+WcRTv8L6cWHYL/a97Auv+nYjFRurrvyFz9WM11xlHRYufodonKDScsxf/CYCd36zj58u2Upx1gpvO68/pI+J1mKfyaC7pwzfGrAHW1Dv3YK2v7wPuc8WzejIvqw17eF+8A3sx8pana85rMnePQROmMmjCVMpKS3j7uy9Z+vg7DIoKYOJgXdNHeSYdeKxUO3nbfRgx8Xwc4yZTWVHB6o1vsvr5r3GUF/P7eeMJC/Z3d4hKAZrwO1VmchJZf76y5thRXop4WQHDDRePBQMnlt2C2HyInOvsNjAOR8316e8vwTgqqCzIZvuTN9acr8j/qQ9fuY/FywuLlxejLnSO9CkpKuDOt57GUVbCkKBiFpwzhPioMDdHqXoyTfidyctK7M0v1RyeWHYL0dc+WdNXX7vP3mJzrvciFkvN9Y7yEqKvfZLyjCR8e/etOX/kyas773tQLebjF8BZV/0fAMcP7OLBD9cQJz8Q6m9n0UWjdAMX1ek04XciQTAVZXXO1d2ARijLOEplYQ7Hnrq65u3eVFZw8LlbcRTlUpZ2CDCUZhytucsiP/1QqD87t/Z55T4xA4YSM2AoxYX5FBXkc9MLjxPmb2NkjD+/nKojfVTn0ITficRiweZdd/x27Vmc1uBILDZvxGLhtPtW1pw/+Nyt/PHF1TxwzXT6Jg49qd3SsPCar3XopWfz9Q/E1z+Qc25wri+459v1XLtsE0U5adw+JZHQID8G9+2ts3tVh9CEr5QbDRw/hYHjp1BeVsqKr9dTcigX+e86Lj9jIFNO1ZE+yrU04bdRcxOcbpsxgQpH3f2CK8tKOfrczWBxTq6qLMymLPUggLOrpqp7xzgq2fTH2TX3mcpybrhoLADWfTsxGKy2n9Z014lXXZ/N286Is50roBYXzuCdz//Lu1u+QEwF/++KsfQO1S451X6a8NuouQlOFQ5Dv1tervPZ4SeuJvySO6Hq1/XU1+4jc+3jGH76wSAIOCqJve2n5HzsyYXE3fofUl67n8wPHqciLwN7cETN5zrxqnvx9Q9k9DTnZPWSokLuXvUkXo4yEvxLmHfWQMKC/QkJ9HNzlKor0oTfiSy+geBlxTvcOcJGrN5EXfNPHOWlWGz2mv899tTViAjGmDp9uZFz/4iIkPTEVTrxqofw8fPnzIX3AHB87w7+sWM36Xu3cE5/X+ZPHqaJX7WKJnyluoiYQacQM+gUHOdfTvrxI9z8wtNEBloZFh3IdReOdHd4qgvQhK9UF2OxWIiMSyDyJueis3u/Xc/1y76mOC+buy4eTGRoIFHhQTrSR51EE76L1N9+0OGopLysFAGs3o0vpWtqCrUOKnLTAINxOEh+8Y6frqmsaORupZwjfRg/hYryMv79yVuUF+VjT/+W2WcOZPKoeHeHpzyI1J344zk8fT38+qNhqrcf9PLxY/h1j7L5z3MAwZhKLNWjcspKES8vwPnmZSrLEa+6qyt6+YcgVm/CLvxp+9/Ulf+v5p7q+8Kjf5ppW3sEjo7SUQD5OZns/3odcmwL3lbh7lljiAoPdndYqjOccWujv9ppwneRxrYfrJ401ZJ7Nv95DrE3v1xTvK2W9MRVPLN2i+uDVj1CaUkxm1f+EzvlxPmXceO0kfjabfj5eDd/s+p6mkj42qWjVDdn9/HlzKudq5Mf37uN+9ZtIvv4PqYPCybY35vpEwZjtXo104rqDjThK9WDxAwaTcyg0TgcDg4e2kthbhZvPr6Cc0f04ZdTR7k7PNXBXJLwRWQasATnjlfPGWMeqfe5HXgZGAtkAlcaYw674tlKqdazWCzEDhgCwOBTz2Dfpk9Y9NynlBYVcPelQ4nv0ws/H28d6dPNtDvhi4gX8BQwBTgGbBKR94wxO2td9ksg2xiTKCJzgb8AV57cWtfVllUq69/jKC8l6YmrTr6wstwlMSrVmAHjzmPAuPOorKhgyZqXKS88SmDhEaaOimVgTC8G9+3t7hCVC7S7aCsipwO/NcZcWHV8H4Ax5s+1rvmw6pr/iYgVSAEiTBMP72pFW6W6m5yMVLLSkjm+ZT0Rlan8asZoYnuHujss1ZwOLtrGAEm1jo8BExq7pmrT81wgDKizVZOILAIWAdz9h0dJnDTLBeEppdoiJDySkPBI+g8bTXlpKQ+s+DsBlv1E+VZyz+xxWETw8rI035DyGB5VtDXGLAWWgr7hK+VJbHY7Z/38AcA50uf65R+Tn5HM3LHhhAfaOWvUAGw60sfjuSLhHwfiah3HVp1r6JpjVV06wTiLt0qpLqZ6pI8xhi0/bqWkMJ9//+MNLhgZw7UX6kgfT+aKhL8JGCgiCTgT+1xgfr1r3gN+DvwPmA180lT/vVLK84kIA0Y492kYPmEy+7Z+xqKlH+KoKOPOS4YyrF+kmyNU9blkpq2IXAz8E+ewzGXGmD+KyO+AzcaY90TEB3gFGANkAXONMQebalO7dJTqmhyVlWx659+UFWQTXJHOvDMSiAoLJiE6zN2h9Qy6tIJSyh0yU0+QdnQf6bu/JYY0gny9uWPWabqsQ0fShK+Ucrfy0lLyczP54a0lDI7w5oE5E3RiV0fQhK+U8iTH927n0FerKcrLYsGESPqGB3LKgGgd6eMKuniaUsqTxAwaScygkRhj+HzzZ5QdKyB79VpG9A1l3MBIzh+d4O4QuyV9w1dKeYTyslLKS0vZ89nbOE78wE0XDmFk/z7uDqvr0S4dpVRX4nA42Pz2M1QW5hBQnsXiKUMIDfQjOkI3cWmWdukopboSi8XC+MtvAiAzOYln92wh8/BuhvjlEhrowzXnn0KAX+Nbh6qGacJXSnm0sKg4wqKck/nzsjPIKcxn8b//Re8gO2P6BnL1+SPcHGHXoV06Sqkua+9Xazi+7VMWTOjD0NhexEWG6kgf7cNXSnVXxhh2ffUhpfnZFOz/H5MG92ZwbC/OPqWfu0NzD034SqmeoKSokILcbI5s/Rhb2k6sYnhg9ljCQwLcHVrn0aKtUqon8PHzx8fPn/BLfg5ASVEBd618nBCTza9njMLf15teQf5ujtJ99A1fKdXtZaUc4/DWj8lNOcaY0GL6hPgz84yBBPn7ujs019MuHaWUcspITqKkqJB9a/9NVLA3o/uFsPC8bjTSRxO+Uko1bM9Xa0nb8Slzx/VhwuBogv19sHblkT6a8JVSqmk/fPouRVmplB77kUtPjSE2PJDTh/d1d1itpwlfKaVapqggn8yUY6Ts2oQtZRu+3lb+72djiOwV5O7QWqajEr6I9AJeB+KBw8AcY0x2A9dVAjuqDo8aY2Y017YmfKWUJygtLmLTikfx9XKQEOTg9kvH4GO3uTusxnVgwv8rkGWMeURE7gVCjTH3NHBdgTGmVQNhNeErpTxN0s7N7Pn8v5wZY+gfGcykU/oRHOBhI306MOHvASYbY5JFJArYaIwZ3MB1mvCVUt3G8YN7KCkq4PAny+kbamNE316es6ZPByb8HGNMSNXXAmRXH9e7rgLYBlQAjxhj3mmkvUXAIoC7//Do2MRJs9ocm1JKdZa936wjb9fnlJcUc/+sEQyICXff9o3tSfgi8hHQ0C4EDwAv1U7wIpJtjAltoI0YY8xxEekPfAKcb4w50NRz9Q1fKdXVVJSXsfndZZQl7+QXZyfQK9CP0YNiOzeI9iytYIy5oLHPRCRVRKJqdemkNdLG8ar/PSgiG4ExQJMJXymluhqrzZuJs2+gMD+XNQd2k/PDbrzXryPE34dbLhnl9g1c2tul8zcgs1bRtpcx5u5614QCRcaYUhEJB/4HzDTG7GyqbX3DV0p1B8YYykpL+PbVvxJkcxAXCHdfPq7junw6sA8/DFgJ9AWO4ByWmSUipwE3GGOuE5EzgGcBB2AB/mmMeb65tjXhK6W6o6Sdm9n96TtMjrcyJiGcYfF9XDvSRydeKaWUZzm863uK8rNJ/vo9hkb6kBgVzAJXrOmjCV8ppTyTw+GgoryMI99/Qc6OjxFHBXdOH87guIi2NagJXymluobKigo2vf0MlUV59DJZLJ4yjMH9IlvegCZ8pZTqejJOHGH/N+sJzPqRPr0CuOqcwcT2Pmnke12645VSSnU94dH9CP/ZdZQUFVJeVsp9qx6nl3cl0YGWNo300Td8pZTqYpJ2byXpm7UU5WVzzRkxnDWiL4H+Ps4PtUtHKaW6H2MMu/63nuTNH3B6/yCiewVwxb3/0i4dpZTqbkSEYWdMZdD4cynMzeGb/du5oonrNeErpVQXZ7XaCA6LIDjs/Cavs3RSPEoppdxME75SSvUQmvCVUqqH0ISvlFI9hCZ8pZTqITx2lE6A3Up4oLe7w1BKqW7DYydeicgiY8xSd8fREl0l1q4SJ3SdWDVO1+sqsXaVOGvz5C6dRe4OoBW6SqxdJU7oOrFqnK7XVWLtKnHW8OSEr5RSyoU04SulVA/hyQm/K/WNdZVYu0qc0HVi1Thdr6vE2lXirOGxRVullFKu5clv+EoppVxIE75SSvUQHpnwRWSaiOwRkf0icq+742mMiCwTkTQR+cHdsTRFROJEZIOI7BSRH0XkdnfH1BAR8RGRb0Xk+6o4H3Z3TE0RES8R+U5EVrs7lqaIyGER2SEi20Rks7vjaYyIhIjImyKyW0R2icjp7o6pISIyuOrvsvpPnojc4e64WsLj+vBFxAvYC0wBjgGbgHnGmJ1uDawBInI2UAC8bIwZ4e54GiMiUUCUMWariAQCW4BZnvZ3Ks4NOv2NMQUiYgO+AG43xnzt5tAaJCJ3AqcBQcaY6e6OpzEichg4zRiT4e5YmiIiLwGfG2OeExFvwM8Yk+PuuJpSla+OAxOMMUfcHU9zPPENfzyw3xhz0BhTBqwAZro5pgYZYz4DstwdR3OMMcnGmK1VX+cDu4AY90Z1MuNUUHVoq/rjWW8kVUQkFrgEeM7dsXQHIhIMnA08D2CMKfP0ZF/lfOBAV0j24JkJPwZIqnV8DA9MTl2ViMQDY4Bv3BtJw6q6SbYBacB6Y4xHxgn8E7gbcLg7kBYwwDoR2SIinjo7NAFIB16o6iZ7TkT83R1UC8wFXnN3EC3liQlfdRARCQBWAXcYY/LcHU9DjDGVxpjRQCwwXkQ8rqtMRKYDacaYLe6OpYXOMsacClwE3FzVFelprMCpwNPGmDFAIeCx9TuAqm6nGcAb7o6lpTwx4R8H4modx1adU+1Q1Se+ClhujHnL3fE0p+rX+Q3ANHfH0oAzgRlVfeMrgPNE5D/uDalxxpjjVf+bBryNs9vU0xwDjtX6je5NnD8APNlFwFZjTKq7A2kpT0z4m4CBIpJQ9RN0LvCem2Pq0qqKoc8Du4wxj7k7nsaISISIhFR97YuzcL/bvVGdzBhznzEm1hgTj/Pf5yfGmKvcHFaDRMS/qlBPVRfJVMDjRpUZY1KAJBEZXHXqfMCjBhU0YB5dqDsHPHA9fGNMhYjcAnwIeAHLjDE/ujmsBonIa8BkIFxEjgEPGWOed29UDToTWAjsqOofB7jfGLPGjTE1JAp4qWrkgwVYaYzx6CGPXUAk8LbzZz5W4FVjzAfuDalRtwLLq170DgK/cHM8jar64TkFWOzuWFrD44ZlKqWU6hie2KWjlFKqA2jCV0qpHkITvlJK9RCa8JVSqofQhK+UUj2EJnyllOohNOErpVQP8f8BS6AnYXokPO0AAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "41O1BacbHXIL",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 69
        },
        "outputId": "9afb8c27-0f80-4fb7-edaf-13d9eeecdba0"
      },
      "source": [
        "%watermark -iv"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "torch 1.5.1+cu101\n",
            "numpy 1.18.5\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    }
  ]
}